Frame-rate Multi-body Tracking for Surveillance

Video surveillance is watching an area for significant events. Perimeter security generally requires watching areas that afford trespassers reasonable cover and concealment. Almost by definition such “interesting” areas have limited visibility distance. These situations call for a wide field of view, and are a natural application for omni-directional VSAM. This paper summarizes our ongoing efforts on developing an omni-directional tracking system. We begin with a few examples and then discuss the background and application constraints. We end with a summary of our approach and its novel components. 1 Examples & Background The paracamera system captures omni-directional video that allows one to generate geometrically correct perspective images in any viewing direction. Figure 1 shows an example. While unwarping in multiple directions and then doing tracking on the perspective images would be possible, it would add considerable expense. Therefore, we are working directly in the complex geometry of the paraimage. While it is acceptable to run tracking algorithms directly on the paraimage, it is not the best way to show the targets to human users. The system provides the user a collection of windows that contain perspectively corrected images. While any number of windows are allowed, we generally use between 1 and 6 depending on the anticipated number of moving objects. The viewing direction within these windows can be controlled via the mouse, or set automatically such that the perspective windows track the N most “significant” targets. Note the “spatial resolution” of the paraimage is not uniform. While it may seem counter intuitive, the This work supported in part by DARPA VSAM program. Figure 1: Tracking system with a single perspective “target” window. spatial resolution of the omni-directional images is greatest along the horizon, just where objects are most distant. While the process scales to any size imager, the current systems use NTSC (640x480) or PAL (756x568) cameras. If we image the whole hemisphere, the the spatial resolution along the horizon is pixels degrees pixels degrees (5.1 PAL) which is 14.3 arcminutes per pixel (11.8 PAL). If we zoom in on the mirror, cutting off a small part of it, to increase the imaged mirror diameter to 640 pixels (756 PAL), we can achieve 10.7 (6.6 PAL) arcminutes per pixel. As a point of comparison, let us consider a traditional “wide-angle” perspective camera. It would take 3 cameras with a horizontal FOV to watch the horizon, but of these each would have degrees pixels degrees , i.e. about the same as the paracamera. Clearly, the traditional cameras would need more hardware and computation. This paper appeared in the 1998 DARPA Image Understanding Proceeding and is copyrighted. Figure 2: Tracking soldiers moving in the woods at Ft. Benning GA. While the lack of motion information and loss of resolution in printing has obscured the details, each box is on a moving target. Every surveillance system must consider the tradeoff between resolution and field-of-view. The paracamera’s unique design yields what may be a new pareto optimal design choice in the resolution/fieldof-view trade-off. We have the horizontal resolution of a 150 camera but cover the full 360 of the horizon. With a wide field of view, objects to be tracked will cover only a small number of pixels. With 4.2 pixels per degree, a target of dimension 0.5m by 2.0m, at 50m will be approximately 2 pixels by 8 pixels, i.e. 16 pixels per person. At 30m, it yields approximately 32 pixels per person, presuming ideal imaging. Realistic tracking in a such a wide field of view requires the processing of the full resolution image with a sensitive yet robust algorithm. Tracking systems abound, e.g., see [Flinchbaugh and Olson-1996, Intille et al.-1997, Wren et al.-1997] and our system draws ideas from these and many other papers. Outdoor operation in moderate to high cover areas restricts the techniques that can applied. Furthermore, we are looking for soldiers not tracking pedestrians in a store or parking lot. Some constraints, and their implications for our systems include: Correlation, template matching and related techniques cannot be effectively used because in a paraimage, image translation is a very poor model; objects translating in the world undergo rotation and non-linear scaling. The lighting is unconstrained. We must handle sunlight filtered through trees and intermittent cloud cover. (We are not considering IR cameras, yet). Targets will probably use camouflage to blend in, so color is not likely to add much information. Figure 2 shows an example scene with solders in the woods. Trees/brush/clouds all move. The system must have algorithms to help distinguish these “insignificant” motions from target motions. Many targets will move slowly (less than pixel per frame); some will move very slowly. Some will try very hard to blend into the motion of the trees/brush. Therefore frame-to-frame differencing is of limited value. Targets will not, in general, be “upright” or isolated. Thus we have not added “labeling” of targets based on simple shape/scale/orientation models. Targets need to be detected quickly, when they are still very small and distant. Since field use will require ruggedized lowpower units, we should use generic computing hardware. 2 LOTS: Lehigh Omnidirectional Tracking